Exemplary embodiments of the invention are directed to a method for segmenting data of a 3D sensor produced in the presence of aerosol clouds for increasing the situational awareness and the location detection of obstacles in order to prevent a loss of the spatial orientation in the event of visual impairment due to the aerosol cloud.
In arid, desert-like areas, such as for example Afghanistan, Iraq or Syria, a strong turbulence of sand and dust particles, a form of aerosol cloud, often occurs during remote landings of rotary wing aircraft, such as for example helicopters. The effect is caused by the so-called downwash of the main rotor or the rotors of the aircraft. The chaotic sand and dust turbulences result in complete or partial loss of pilot vision outside the cockpit—the so-called brownout. Other forms of aerosol clouds such as whiteouts (turbulent snow during the landing), smoke, or fog hinder the view and can also significantly restrict the situational awareness of the pilot in a hazardous manner. Due to the absence of a view or the limited external cockpit view there is a risk of the loss of spatial orientation above the ground, in particular with respect of pitch and roll angles as well as unintended lateral drift of the landing aircraft. Moreover, location detection of obstacles in the landing zone is severely limited. All of this increases flying accidents.
Actual sensor data for the landing zone are required to enable a synthetic spatial view and orientation aid for the pilot for maintaining the situational awareness and the location of obstacles. For this purpose, different systems (for example radar, laser, camera systems, GPS etc.) are used.
German patent document DE 102009035191 A1 describes a radar sensor applied to a synthetic display of the surroundings, which in the event of a brownout is supplied with additional data of a radar sensor that is activated for this purpose.
With radar systems, however, significant problems can occur due to so-called crosstalk during the measurement of the landing area that is only a few meters away with simultaneous strongly varying pitch angles during the final landing process as well as due to echoes from side lobes.
Laser sensors have a much higher spatial resolution compared to radar systems due to the short wavelengths thereof of, for example 1.5 μm, and are therefore considerably better suited to detecting important details of the situation environment as well as hazardous obstacles (such as for example high voltage lines) in the landing zone of a helicopter. However, laser sensors, as optical systems in contrast to radar systems, can often not fully penetrate a brownout-cloud, because the laser pulses are already reflected back to the sensor, scattered or absorbed by parts of the turbulent dust cloud. In the received laser measurement data, in general parts of the brownout cloud conceal the free view of the landing area lying behind the cloud and any obstacles that may exist.
The physical property of laser sensors makes them appear superficially as less suitable for assisting pilots during brownout landings.
U.S. patent document US 2011/0313722 A1 discloses a method based on laser sensor data with which a correlation of the falling edge of a laser echo with a threshold value takes place, wherein there is a measurement technology difference between obstacles and aerosol clouds.
In known numerical calculation methods, by which turbulent dust of a brownout cloud can be segmented, a global accumulation of all sensor measurement data from multiple numbers of complete (laser) sensor recording cycles (so-called sensor frames) is carried out. During this, very large amounts of data are accumulated. Following the accumulation, it is attempted to determine statistical properties of the measurement points that should enable dust measurement points to be distinguished from real static obstacle measurement points.
The disadvantage of this type of method is that conclusions regarding local properties of individual isolated dust measurement points are based on a very large, global database by means of the accumulation of all measurement points of multiple sensor frames. This may result in very large computation efforts and inefficient processing time.
Exemplary embodiments of the present invention are directed to a method for more efficient and more precise detection of measurement points of an aerosol cloud in real time based on laser sensor data for (significantly) increasing the situational awareness and the location detection of real obstacles.
The method according to the invention for segmenting the data of a 3D sensor produced in the presence of aerosol clouds in order to achieve an increase of the situational awareness and the location detection of obstacles includes the following process steps/processing steps:                1. Transforming the sensor data into a 3D measurement point cloud,        2. Determining connected subsets of the 3D measurement point cloud, so-called measurement point clusters, based on the local measurement point density. This step is performed based on the sensor data of a single measurement cycle of the 3D sensor,        3. Determining at least one of the following characteristic parameters of the individual measurement point clusters determined in step 2:                    position,            orientation in space,            shape                        4. Determining the variation with time of the characteristic parameters using the sensor data recorded in subsequent measurement cycles, from which the association of a measurement point cluster with a real obstacle or with the aerosol cloud results.        
The present method for segmenting sensor data for increasing the situation awareness, in particular of a vehicle driver, and the location detection of obstacles within an aerosol cloud (for example a brownout cloud) is preferably performed in combination with the use of a laser sensor system, wherein such a system can comprise, for example, the following components: a 3D laser sensor for detecting obstacles, an electronic data analyzer for the recorded measurement cycles (so-called sensor frames), and an output device (for example a display screen), wherein the system or parts thereof can be integrated within other systems or can collaborate with other systems by transferring and exchanging or transmitting and receiving suitable data.
The method according to the invention enables reliable detection of real obstacles within the scanned aerosol cloud/turbulence.
The invention can be used for all the situations mentioned in which there is a visual impairment/restriction of the external cockpit view by dust, smoke or fog or turbulence of the elements, including for example known phenomena such as brownouts (dust/sand turbulence) or whiteouts (turbulent snow).
It is irrelevant to this whether the brownout situation is caused by the rotor downwash of a landing rotary wing aircraft or aircraft with vertical takeoff and landing capability or by natural phenomena (i.e. conditions similar to brownout), such as wind or other weather effects or even by the movement of other (airborne) vehicles.
The invention will be described below using a brownout situation representative of all forms of aerosol clouds.
The method according to the invention is based on the numerical analysis of high-resolution 3D data. The 3D data are advantageously recorded in real time before and during the brownout landing by a laser sensor that is typically mounted on the helicopter (such as for example a SferiSense® sensor of the Airbus Defence and Space GmbH, Ottobrunn, Germany), wherein the use is not limited to flying or moving vehicles, but is also possible in static systems.
The methods of the present invention provide reliable detection of turbulent dust or sand of the brownout cloud from the 3D measurement data of a laser sensor, and hence provides for the segmentation of the same from real obstacles, wherein the segmentation is carried out using cluster formation and characteristic parameters of those clusters. The discrimination of the association of a measurement point with the brownout cloud is performed by the analysis of the variation with time of the cluster parameters. Due to the special form of processing for the dust segmentation, the disadvantage of laser sensors during brownout landings is negated and looking through the dust cloud can be practically carried out, which advantageously results in significantly increasing the situational awareness, in particular for a pilot, and the location detection of obstacles.
The calculation method according to the invention and described in detail in the figures reverses the logic of known methods. The basis of those methods is a global accumulation of all sensor measurement data from multiple numbers of complete recording cycles of the sensor field of view (FOV). Due to the reversal of the processing logic of known dust cluster calculation methods and systems from global→local to local→global, a significant efficiency gain results for the processing of the 3D data.
The procedure enables very computationally efficient processing and accurate, practical frame-accurate calculation results to be obtained, whereas the known methods require the accumulation of 3D data over a number of multiple sensor frames for their mathematical analysis. In this respect the present invention represents a completely novel approach to the solution of the problem of aerosol/dust cloud detection and segmentation using laser sensors.
Thus, a real-time capable avionic system for pilot support that is suitable for operational use is provided, which facilitates helicopter landings, especially under brownout/whiteout conditions, and significantly reduces the risk of accidents.
The use of the method according to the invention is not however restricted to aircrafts. A corresponding system can also be advantageously implemented in other vehicles or even at static positions. The use of the information obtained with the method can be carried out by a vehicle driver or a machine, for example an autonomous system.